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Abstract 

We present a new interior-point potential-reduction al- 
gorithm for solving monotone linear complementarity 
problems (LCPs) that have a particular special struc- 
ture: their matrix M £ R" x ™ can be decomposed as 
M = $[/ + Ho, where the rank of $ is k < n, and n 
denotes Euclidean projection onto the nullspace of $ T . 
We call such LCPs projective. Our algorithm solves 
a monotone projective LCP to relative accuracy e in 
0(y/nlii(l/e)) iterations, with each iteration requir- 
ing 0{nk 2 ) flops. This complexity compares favorably 
with interior-point algorithms for general monotone 
LCPs: these algorithms also require 0(y/n\n(l/e)) it- 
erations, but each iteration needs to solve an n x n 
system of linear equations, a much higher cost than 
our algorithm when k <C n. Our algorithm works 
even though the solution to a projective LCP is not 
restricted to lie in any low-rank subspace. 

1 Linear complementarity problems 



The LCP for a matrix Met" x ' 
is to find vectors x, y £ R™ with 



and a vector q e 



x > y > y = Mx + q x T y = (1) 

We say that vectors x, y are feasible if they satisfy the 
first three conditions of (p} (i.e., leaving off comple- 
mentarity) , and we call them a solution if they satisfy 
all four conditions. The complementarity gap x T y is 
nonnegative for any feasible point (x,y), and measures 
how close a feasible point is to being a solution. (See pQ 
for an overview of LCPs.) 



"This work was first presented at the 2010 NIPS workshop 
"Learning and Planning from Batch Time Series Data." 



If M is positive semidefinite (but not necessarily sym- 
metric) , the LCP is monotone, and there exist interior- 
point algorithms that solve it to relative accuracy e in 
0(y/n\n(l/e)) Newton-like iterations. In each itera- 
tion, the main work is to solve an n x n system of 
linear equations. 

Suppose the matrix M can be decomposed as M = 
<S>U + n , where <t> e W ixk and U G K fcx ™ have rank 
k < n, and ITo projects onto the nullspace of $ T (that 
is, Hq = I — $<&T ) where t denotes the Moore-Penrose 
pseudoinverse) . In this case we call (M, q) a projective 
LCP of rank k. Our new algorithm solves a projective 
LCP in 0(-y/nln(l/e)) iterations, the same as for the 
general monotone case, but with each iteration requir- 
ing only 0(nk 2 ) flops. 

This result is an analog of the situation for linear equa- 
tions: a rank-Zc factored system of linear equations can 
be solved in 0(nk 2 ) flops, while it is believed that a 
general n x n system of equations requires Sl(n 2+ ' ; ) 
flops for some constant n > 0. However, unlike the 
situation for linear equations, in a projective LCP we 
can't a priori restrict either x or y to a low-rank sub- 
space of R n - so, it is perhaps surprising that the anal- 
ogous complexity result still holds. (The inequality 
constraints x > and y > are the source of this 
difficulty: the intersection of x > or y > with a 
rank-fc subspace can be quite restrictive.) 

2 Potential reduction 

We say that x and y are strictly feasible if they satisfy 
x > 0, y > 0, and y — Mx + q. We will assume that 
we know a strictly feasible initial point (xo, yo) for our 
LCP. (If we do not, it is possible to construct one, as 
mentioned in 151.) 
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For any strictly feasible point (x, y), fixing a parameter 
K > 0, we define the potential 

n n 

Pk(x,IJ) = (n + K)lnx T y -^bur, -^lnj/j (2) 

»=l i=i 

We will design an algorithm that attempts to reduce 
the potential p K (x, y) over time. The following lemma 
justifies this idea: 

Lemma 2.1 For any strictly feasible x,y, 

n n 

Po(x,y) = n\nx T y -yfan -^Inyt > (3) 

i=l i=l 

(For a proof, see the appendix.) In particular, 
Lemma 12. f I implies p K (x,y) > Hclnx T y, or x T y < 
exp(p K (x,y)/K); so, if we can reduce the potential by 
at least some amount S > per iteration, after T iter- 
ations our iterate (x,y) will satisfy 

x T y < exp((p K (x Q ,y Q ) -T5)/k) 

That is, our algorithm will converge linearly: a bound 
on the gap will decrease by a factor of exp(— 6/n) per 
iteration. Below, we will take k = yfn, and d will 
not depend on n; so, if we desire a reduction of our 
potential by a factor < e < 1, we will need ln(l/e) 
iterations, as the abstract states. 

3 The central path 

Lemma [2.f I shows that po(x,y) > 0. The local mini- 
mizers of po are the points where its gradient vanishes: 

= itiPo( x >y) = n Vi/ xT y - l l x i 

= ^Po{x,y) = nXi/x T y - l/y. t 

Multiplying the first equation through by Xi(x T y)/n, 
or the second equation through by yi(x T y)/n, we get 

XiVi = x T y/n 

which is satisfied for a pair x, y > if and only if 
Xiyi = t for all i and some t > 0. Equivalently, we 
can write x o y = tl, where o denotes the Hadamard 
(elementwise) product. 

The points (x, y) that satisfy 

x > y > y = Mx + q x o y = tl 

are called the central path of the LCP (M, q); for mono- 
tone LCPs, if the central path is nonempty, it is a 
smooth curve, and it approaches the solution of the 



LCP as t — > 0. We can view the term po(x,y) as en- 
couraging our algorithm to remain close to the central 
path; the remaining part of our potential, K\n(x T y), 
encourages our algorithm to slide along the central 
path, reducing t — x T y/n and pushing us closer to 
a solution. 

4 A result on rank 

To make our algorithm run in time 0(nk 2 ) per iter- 
ation, we will need to do most of our calculations on 
vectors of length k instead of length n. Unfortunately, 
as mentioned earlier, the vectors x and y are not guar- 
anteed to lie in any rank-fc subspace. Our main insight 
is that we can work mostly from a function of x and y 
that does lie in a rank-fc subspace. In more detail: 

Lemma 4.1 Suppose the pair x,y is feasible for the 
monotone LCP (M, q), and that M = $[/ + IIo, where 
$ has rank k and IIo projects onto the nullspace of $ T . 
Then the vector x — y + q is in the range of 

Proof: Define n = I — Hq, so that the range of II 
is the same as the range of Since n projects onto 
the nullspace of $ T , we know Hq$ — 0. And, since 
il is a projection matrix, we have Ilg = ilg. So, 
n M = n ($t/+n ) = n . Therefore, for any feasible 
x,y: 

y = Mx + q 

n y = UqMx + n oq 
IL y = U x + U q 
(I - U)y = (I - n> + (7 - TS)q 
Il(x - y + q) = x-y + q 

So, x — y + q is in the range of If, as claimed. □ 

In general, we can't recover x or y individually from 
x — y + q. However, if we know that the pair x, y solves 
the LCP, we can use complementarity to recover x and 
y: x T y = 0, so for any i, at most one of Xi and yi can 
be nonzero. So, given x — y + q, we subtract q to get 
z = x — y. Then we set x — z + and y — 2_, i.e., 
Xi = max(zi, 0) and ?/, = max(— z%, 0). 

At intermediate points in our algorithm, we maintain 
x and y separately, and constrain x — y + q = $>w. 
We calculate the update for w first by manipulating 
length- k vectors, and then use this result to derive 
updates for x and y with work that is only linear in n. 

5 The algorithm 

We will base our algorithm on a potential-reduction 
method due to Kojima et al. [2 . The algorithm 



In: LCP (M, q); strictly feasible x,y; (3 £ (0, 1); e > 0. 
Out: strictly feasible x, y with x T y < e. 

1. Stop if x T y < e. 

2. Solve (gHSJ) for an update (Ax, Ay). 

3. Choose a step length & > by 0. 

4. Set (x, y) <- (x, y) + 6(Ax, Ay). 

5. Repeat from step 1. 



Figure 1: Potential-reduction algorithm. 

(Fig. [T} uses Newton's method to step toward a point 
on the central path with x o y = tl. It has a single 
parameter /3 € (0, 1) that helps us choose the target 
complementarity t: we set t to be a fraction f3 of the 
current average complementarity, i.e., t — /3^-^-. Tak- 
ing /3 near 1 causes us to follow the central path closely, 
reducing t slowly; taking j3 near tries aggressively 
to reduce the complementarity gap, but risks straying 
farther from the central path 



We will derive the algorithm first for general M (in 
which case it is the same as that of Kojima et al.), and 
then specialize it to the projective case M = $J7 + IIo. 
We will assume that M is positive semidefmite, i.e., 
that is positive semidefinite. 

To step toward a point on the central path with com- 
plementarity t, we want to satisfy both x o y = tl and 
y = AIx + q. The first-order Taylor approximation to 
x o y = tl is 

x o y + Ax o y + x o Ay = tl 

Writing X — diag(x) and Y — diag(y), the Newton 
step is therefore 



Y X 
-M I 



Ax 
Ay 



where 



9 = P^ 1 - x ° V 



; g v - 

n 

Mx + q — y 



(4) 



(5) 



Here we have used t 
any feasible x, y. 



>xy 



Note also that r — for 



1 For simplicity of notation we require x > and include 
only inequality constraints, but the algorithm works with minor 
changes if we allow some components of x to be free instead of 
nonnegative, in which case the corresponding constraints are 
equalities instead of inequalities (i.e., the corresponding ele- 
ments of y must be zero). 



To pick a step length, we first define a vector s and 
diagonal matrix S with 



&i — Sa — \J Xiyi 



(6) 



Write so for the smallest element of s. Like g, the 
vector s/sq measures how far we are from the central 
path: if we are near the central path, s/sq 1, and we 
can afford to take a relatively larger step, while if we 
are far from the central path, some elements of s/sq 
will be large, and we will need to be more cautious. 



In particular, we will show below that the step size 



3 s 
7 llS-if 



(7) 



guarantees that we maintain strict feasibility and de- 
crease our potential. Note that s > for any strictly 
feasible x,y; and, since f3 < 1, we have j / for 
any strictly feasible x, y. So, (J7]) always yields a well- 
defined step length 9. In practice, the value of 9 
from J7]) will be conservative; but, it could serve as 
an initializer for a line search (e.g., [2 Alg. 9.2]) to 
determine a step length that decreases the potential 
as much as possible. 

6 Proof of correctness 

We proceed to show that the potential-reduction al- 
gorithm behaves as claimed above. The proof follows 
Kojima et al. [2], although our presentation is some- 
what different. 

Lemma 6.1 Eq. ^ defines a unique step direction. 
For a proof, see the appendix. 

Theorem 6.1 Suppose n > 2, and take the parameter 
k from our potential @) to be n = ^/n. Suppose x and 
y are strictly feasible for the monotone LCP (M,q). 
Then the step (9 Ax, 9 Ay) from gfQ), with (3 = 
maintains strict feasibility and guarantees a potential 
reduction of at least: 



p K {x + 9 Ax, y + 9 Ay) < p K (x, y) 



Proof (Thm. 16. ip : To make notation simpler, we 
will change variables tou = X~ x Ax and v = Y~ 1 Ay. 
With this notation, and using ©, the first row of (0| 
becomes 

S 2 (u + v) = g (8) 
Our goal is now to bound the change in potential 
Ap = p K (x + 9 Ax, y + 9 Ay) -p K (x,y) 



We start by splitting p K (x,y) into two pieces, so that 
we can bound each piece separately: 

P K (x,y) =pi(x,y) +p 2 (x,y) 

n n 

P2(x,y) = -^lnxj - J^lny l 

i=l i=l 

Here we have used n + K = jj, from the assumed value 
of (3. Note that pi is convex, but pi is not (due to the 
concave In function and the interaction x T y). Write 

Apx =p 1 (x + 9 Ax, y + 9 Ay) - pi (x, y) 
Ap 2 = p2(x + 6Ax,y + 6 Ay) -p 2 {x,y) 

To upper bound Api we will use the identity 

ln(z + Az) < In z + Az/z 

which holds since ln(z) is concave in z. We take z = 
x T y, so 

Az = (x + 9Ax) T (y + 9 Ay) - x T y 



= 9 Ax 1 y + 9x 1 Ay + 9 2 Ax T Ay 



Therefore, 



Api < ^{9Ax T y + 9x T Ay + 9 2 Ax T Ay] 
or, in terms of u and v , 

Api < ^[9(xo y y(u + v) + 9 2 1 } 

where we have written 7 = Ax T Ay — u T XYv. Note 
that 7 > 0, since 7 = Ax T Ay — Ax T MAa; and M is 
positive scmidcfinitc. 

For Ap2 we use a local upper bound on — ln(z + Az), 
derived from the second-order Taylor approximation 

-ln(z + Az) « -In 2 - Az/z+ \{Az) 2 / z 2 

To get a bound valid for some range of Az, we scale 
up the second derivative by a factor r > 1: 

Lemma 6.2 For r > 1. if ^ > 1 ^ L . then 

-ln(z + Az) < -lnz- Az/z + §(Az) 2 /z 2 (9) 



(See the appendix for a proof.) So, using Lemma 16.21 
2n times (first with z — Xi and Az = 9Axi, and then 
with z = yt and Az = 9Ay{), we have 

n n 

Ap 2 < & x i/ x i - e J2 A y*/ y * 

i=l i=l 

n n 

+ 9 2T -Y^(Ax l ) 2 /x 2 + 9 2T -^m) 2 /yl 



so long as 9Axi/x,i > and 9Ay.i/yi > for all 
i. Or, in terms of u and v, 

Ap 2 < -91 T {u + v) + ^9 2 (u T u + v T v) 

as long as 



9u > 



9v > 



(10) 



Combining the bounds on Api and Ap 2 , we have 

A P < 1 £ r -[9{xoy)T(u + v)+9 2 1 ) (11) 
- 91 T {u + v) + ^9 2 (u T u + v T v) 

as long as (fTUf holds. Using the definition of g, we 
can split the right-hand side of (TTTI) into a term that 
is linear in 9: 

-^9 T (u + v) 
and a term that is quadratic in 9: 

d 2 [^l+i(u T u + v T v)} 

We can simplify each of these terms separately: us- 
ing ((HI), we have 



g , (u + v)=g T S- 2 g=\\S-'g\ 



And, 



+ v T v=\\S- 1 Su\\ 2 + \\S- 1 Sv\\ 2 
<jz(\\Su\\ 2 + \\Sv\\ 2 ) 

= jz(\\S(u + v)\\ 2 -2 1 ) 

= 4(||5- 1 5 || 2 -2 7 ) 



(The second line holds by definition of sq; the third 
uses the definition of 7; and the last uses ([5]) again.) 



So, (fTT|) becomes 



Ap<9 2 [(^ T - y ~^) 1 + ^\\S^g\\ 2 ] (12) 
-^ll^ffll 2 



as long as flT0|) holds. In Lemma 16.21 we are free to 
choose t > 1; so, we will assume 



So, since 



1 



x y_ s s \, 



(13) 



; = 1 



i=l 



we have that (~\ ir)7 < 0, and (1T21) becomes 

pX y Sq 

Ap^^^ll^^H 2 -^^- 1 ^! 2 (14) 

as long as (ITUl) and ([13")) hold. The right-hand side will 
be negative for the optimal 9 > 0, since its derivative 



with respect to 9 is negative at 9 — 0. So, we now know 
that Ap < 0, i.e., (Ax, Ay) is a descent direction for p 
as desired. 

To determine how large a decrease in potential we can 
achieve, we need to pick a feasible step size 9. To 
ensure that we satisfy (fTU)l . we will enforce the stricter 
constraints 



N|oc<^ |Hloo < ^ 



(15) 



Note that (TTSj) implies that our step maintains strict 
feasibility: since 2-^- < 1, we have H^Hoo = 
pX^AxWoo < 1 and ||HU = pY^AyW^ < 1. 
Now, 

IMloo < IMI 

< -\\Su\\ 



< ^(11^11 + 11^11+27) 
= j-JSu + Sv\\ 

= ^11^11 



Analogously, 



So, (fl~5j) will be satisfied if we take 

T \\S 1 g\\ 
Substituting into (fl4|) . we have 

(r-l) 2 sg t ||C-1„||2 



(16) 



_ r-l sq n II c-1 || 2 

r IIS-^H ^T^H D -911 

= |^-^ i ^jl|5- 1 9 || (17) 

as long as (fl~3)) holds. Finally, we lower-bound 1 1 iS' 1 I i 
with the following lemma, whose proof is in the ap- 
pendix: 

Lemma 6.3 For g in f5j), if f3 — n _^^ , then: 

\\ S -lg\\ > ^sl±A 
11 & \\ — 2 n so 

Substituting Lemma 16.31 into (I17p , we have 
Ap<\^-^ 

In particular, if we take r = |, we satisfy (ITBl : n > 2, 
so i < ra x 707 ^ And; we have 



Ap < 



1_9_4 _ 3 73 
2 16 7 7 2 



< 



as claimed. This value of r, together with (|16l) . 
yields Q. □ 



In: $, /7, g; strictly feasible x,y; /? e (0, 1); e > 0. 
Out: strictly feasible x,y with a; T y < e. 



Set ui ^(cc - y + q). 
Stop if x T y < e. 

Compute 5 and r from ([5]) and (|24|) . 
Compute G and /i via (12"2TJ2"3T) . 

5. Solve GAw = h for Aw. 

6. Solve (X + Y)Ay = g - Y$Aw for Ay. 

7. Compute Air — Ay + $Aw. 

8. Choose a step length 9 > by ©. 

9. Set (x, y, uj) <r- (x, y, w) + 9(Ax, Ay, Aw). 
10. Repeat from step 2. 



Figure 2: Potential reduction for projective LCPs. 



7 Algorithm for projective LCPs 

The main work in each iteration of the potential- 
reduction algorithm is to compute the Newton direc- 
tion Q , which requires solving annxn system of lin- 
ear equations. (The system ^ as a whole is 2n x 2n, 
but we can use the sparsity of the three diagonal blocks 
to eliminate cheaply down to an n x n system.) The 
work required to solve this n x n system can vary 
greatly, depending on the structure of M, but is of- 
ten prohibitive for large n. 

So, in the projective case (M = <E>£7 + Ilo and IIo = 
J_$$t ; w here $ has k < n columns), we want to avoid 
solving annxn system at all; instead we will construct 
and solve only a smaller k x k system. Constructing 
the k x k system will then be the main work in each 
iteration, at 0(nk 2 ) flops. (Solving the k x k system 
takes at most 0(k 3 ) flops even if we just use simple 
Gaussian elimination.) 

To run our potential-reduction algorithm on a projec- 
tive LCP, our basic idea (as discussed in Sec. HJ is to 
keep track of w such that x — y + q = $w, and do as 
many calculations as possible in terms of w instead of 
x and y. Fig. [2] summarizes the resulting algorithm. 
(In fact it is not even necessary to keep track of w ex- 
plicitly, but Fig. [2] makes w explicit for clarity.) For 
convenience we assume that <!> has full column rank; if 
not, we can drop some columns from $ and adjust U 
accordingly. 

We are given a strictly feasible pair x, y to start, so our 
initial w is just & (x — y + q) ; we then have x — y + q = 
Qw by Lemma 14.11 We update w by adjoining the 
equation Ax — Ay = QAw to the system With 



this extra constraint, (HJ becomes: 




(18) 



Note that the extra constraint does not change the se- 
quence of points (x, y) visited by our potential reduc- 
tion algorithm: its only effect is to allow us to track w 
and solve ([18)) efficiently 



To solve (|T8|) efficiently, we will run several steps of 
block Gaussian elimination analytically. First use the 
last block row of ([TBI) to eliminate the first block col- 



X + Y Y3> \ ( Ay 
I-M -M$ I { Aw 



(19) 



Since k < n, the total time per iteration is 0(nk 2 ), as 
claimed earlier — potentially substantially faster than 
an iteration of the potential reduction method on an 
arbitrary monotone LCP. Since we are performing the 
exact same sequence of updates to x and y as the gen- 
eral potential- reduction algorithm running on (M, q) , 
our bounds from Sec. [6] continue to hold: we take the 
same number of iterations and reach the same final 
error level. So, we have proven: 

Theorem 7.1 Suppose n > 2, and take the parameter 
k from our potential 0) to be k = \pn~. Suppose x and 
y are strictly feasible for the monotone projective LCP 
(M,q), where M = $U+n 0> U = and $ has 

full column rank. Then the algorithm of Fig. with 
(3 = , maintains strict feasibility and guarantees a 
potential reduction of at least | per step. 



Then note that strict feasibility implies that X + Y is 

nonsingular. So, we can use the first block row of (fT9|) Acknowledgements 
to eliminate the first block column: 



[(M -I)(X + y)-!y$ - M®]Aw = 

(M - I)(X + Y)~ 1 g + r 



(20) 



Finally, we can left-multiply (|20|) by $ T to reduce to 



GAw 



(21) 



where 



G 



$ T [(M - I)[X + Y)^Y - M]$ 

^ ($ T $t/_$ T )(x + y)- 1 y$-$ T ($c/)$ (22) 

h = $ T [(M - I)(X + yr\g + r] 
= ($ T $t/-$ T )(X + y)" 1 .g + $ T r (23) 



Eg s . I2"2l [2"51 show how to build G and h in time 0(nk 2 ), 
starting from x, y, r, g, <& T <i>[/, and <£", which together 
require 0(nk) storage. The vectors r and g can be 
calculated efficiently using (J5j> and the representation 
M = $U + I - we compute 



This work was supported by ONR MURI grant num- 
ber N00014-09-1-1052. 
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A Proofs of Lemmas 

Proof (Lemma 12. ip : Define Ui 
J2i u i- Then x T y = u, and 



yiXi and let u 



Mx = $(Ux - &x) 



(24) 



For the term &X, it may help to precompute a factor- 
ization such as the QR decomposition of $. 

We can then solve (|2"Tj) for Aw in time 0(k 3 ) or better. 
(Lemma 17. 1[ whose proof is in the appendix, ensures 
that Aw is uniquely determined.) Since X + Y is non- 
singular, we can use the first block row of (fH?]) to solve 
for Ay in time 0(nk). Finally, we can use the last 
block row of (fT5|) to solve for Aa; in time 0(nk). 

Lemma 7.1 If x,y > 0, <& has full column rank, and 
is positive semidefinite, then the matrix G defined 
in nffl) is invertible. 



n In x T y~ In Xi — In j/j 
t=i i=i 

71 

= — In Ui /u > 



□ 



since < Ui/u < 1 for all i. 

PROOF (Lem. 16. ip : Since x > 0, X is invertible. So, 
we can use Gaussian elimination on Q to arrive at 

(-M - X- l Y)Ax = Mx + q-y- X^itl -xoy) 

(In particular, subtract X^ 1 times the first row from 
the second row). 



Since M is positive semidefinite and x,y > 0, M + 
X~ X Y is strictly positive definite, and so we can solve 
uniquely for Ax. We can then substitute Ax into the 
first row of ij^j). which leads to a unique solution for 
Ay since X is invertible. □ 



PROOF (Lemma I6.2J) : By construction, the left-hand 
and right-hand sides of (0) match in value and first 
derivative at Az = 0. The second derivative of the 
left-hand side with respect to Az is (z + Az)~ 2 , while 
that of the right-hand side is tz~ 2 . When Az > 0, 



tz > (z + Az)~ 
holds as long as 



holds. When Az < 0, 



(z + £)- 2 d£< 



2 dt 



I Az 
1 f 



< 



- 2 Az 



z + Az z 

-Az < -t(1 + ^)Az 

l<r(l + ^) 
I _ 1< A* 



and f (the zth diagonal element is yij (xi + yt)). Since 
<t> T lIo = 0, we can rewrite the first line of (|22|) as: 

G = $ T [$[/(£> -/)-£>]$ 

The matrix — D) is positive semidefinite: it has 

the same eigenvalues as its similarity transform 

(I - D)^HU(I - D) 1 / 2 

which is positive semidefinite since is of the form 
X T AX for real matrices A and X with A = <&U posi- 
tive semidefinite. The matrix D is strictly positive def- 
inite, since it is diagonal with strictly positive diagonal 
elements. So, the sum $[/(/ — D) + D is also strictly 
positive definite, as is <I> T (<f>U(I - D) + = -G 
since $ has full rank. So, G is invertible as claimed. 
□ 



(In the third line from the bottom we multiply through 
by z(z + Az) > 0, and in the second line from the 
bottom we divide through by — Az > 0.) □ 

Proof (Lemma [673]): Write £ = S~ l l, i.e., & = l/s<. 
Write u = — -. We have: 

~ 77. 



lls-WHI/^-sf 

= ||/3(/i£- S )-(l-/?) S || 2 

= II/3(m£-s)II 2 + II(i-/^II 2 

since — s) T s = n/i — s T s = 0. The first term is 
a sum of squares, so is at least as large as any of its 
components: 



\S- 1 g\\ 2 >^±-s ) 2 + (l-^s T s 



since 1-/3 = so n^i^ = = 1. Finally, 

we can complete the square of (jjti — Sq)> getting: 

||5- 1 5 || 2 >§[| Ai 2 + (i A1 - s 2 ) 2 ] 

- 4M ^ 



as desired. 



□ 



Proof (Lemma EH): Let D = (X + Y)- 1 Y. Note 
that D is diagonal, with all elements strictly between 



